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ON A SHORT METHOD OF LEAST SQUARES. 

By Btjbton H. Camp. 

1. Introduction. In one of the "Scientific Papers of the Bureau of 
Standards"* it is shown that, when the independent variables of a set of 
observational equations are equi-spaced, certain simplifications are available 
which greatly shorten the least square solution. These simplifications are of 
the same character as those which have been used by statisticians in computing 
the coefficients of regression,! and the trend in a time series.| They may be 
described in a particular case as follows: 

Consider the type equation : 

(1) Z= A + BX+CT, 

where X and T are independent variables, giving n equations of condition to 
determine A, B and C. To obtain the first normal equation, add the equations 
of condition. To obtain the second, an-ange them in order of ascending X's, 
multiply the successive equations through by the successive teims of any 
ascending, equi-spaced sequence of integers, and then add the results. To 
obtain the third normal equation, arrange in order of ascending Fs, multiply 
by a sequence of integers, as before, and add. It is sometimes supposed 
necessary that the sequence of integers shall be special in form, but this is in 
fact immaterial, all ascending, equi-spaced sequences leading to the same 
results. It will usually be convenient, however, to choose the sequence: 

(2) — n+1, -n + 3, ..., -1, +1,..., +«-l, 
when n is even; and the sequence: 

(3) - (n — l)/2, - (n — l)/2 -f 1, . . ., 0, . . ., (w — l)/2, 



* No. 388, Adjustment of parabolic and linear curves to observations taken at equal 
intervals of the independent variable, by Harry M. Roeser, July, 1920, Department of 
Commerce, U.S.A. 

t E.g., Yule, Theory of statistics, Griffin and Co., Ltd., 1911, pp. ISlff. 

X E. g., Persons, The Review of economic statistics, Preliminary Volume I, 1919, p. 13, 
foot-note 1. 
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when n is odd. These sequences will be assumed in the calculations to follow 
and in the resulting expressions (10) for the errors. It will be shown that, if 
the X's and the F's are not equi-spaced but are approximately so, this short 
method may be used with negligible error, and expressions for the error will 
be found. It often happens that only approximate equi-spacing is possible. 
One may have only partial control of the independent variables, X, Y, and it 
would be very difficult or impossible to choose them exactly equi-spaced, as, 
for example, when selecting a star list for time observations with a transit. 
Sometimes, when one does have apparent control of these variables and has 
set his instrument so that they are equi-spaced, the spacing is disturbed after 
allowance is made for instrumental errors ; the same efEect is produced if the 
obseiTations have slightly varying weights. It is, therefore, desirable that an 
expression for the error in using the short method should be obtained, and it 
should be in such a form that it may be computed quickly, after the short 
determination is made. Further, it should be possible to estimate roughly the 
size of the error to be expected, before any computation has been made. 

If the order of the Z's is the same as the order of the F's the method as 
outlined above fails. An important case where this happens is when F= X' 
in the type equation. It is feasible to make special arrangements in such cases, 
but they will not be discussed here. The results in the special cases, A, B, orC 
equal to zero, will be given, however. The computer may use the summary 
in § 3 below without reading the theory in § 2. 

2. Theory. Let i be the representative integer of the sequence (2), or (3), 
while the second normal equation is being foi-med, and j' the representative 
integer for the third normal equation. Let Zo denote the middle X, when n 
is odd, the point half way between the two middle Z's, when « is even. Let 
h denote the "normal" space interval of the X's, this to mean that value 
which will make the sum of the squares of the eiTors (e) made in writing, 
X — Xo = h i, a minimum. For the Ps, let Fo, Ic, and d be the analogous 
letters. Let r- denote the residual, Z — (,A-^BX-\-CY). Thus, 

Zt* Zf 

The true normal equations, resulting from an exact application of the least 
square method may be written 

(5) Zr = 0, ZQii + e + Xo) r = 0, Z(kj + d-^ Yo)r = 0, 

and, by virtue of the first of these equations, the ^ Xo r and ^ Fo r of the 
second and third drop out. Make the transfoi-mations 
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(6) X=-x-\-X, Y=y+Y, Z=z-^Z, where Z^ = Z y = Z^ = 0. 
Equations (5) become, after reduction, 

Z = A-\-BX-^CZ, 

(7) ^hiz = B Zhix + C ^kiy — ^er, 
Zkjz = B Zkjx -VC'Zilijy—'Z, dr. 



Solving by determinants the last two of (7), one gets the true values of B and 
C; putting e = (i = in the results gives the values of B and C that would 
be obtained by the approximate method under consideration. The differences 
will be denoted by d£ and dC, and are 



dB = 



1 



(8) 



Zer 


h'Ziy 




2dr 


A = hJ, 


I 



dC = ~ 



2^ix Ziy 



h^ix ^er, 
k^jx ^dr, 



It is now necessary to iind approximate values of these expressions in terms 
of quantities which will be found in the course of the short method compu- 
tation, or of other known quantities. 

Let Sj^j Sj5^ Sq refer to the mean square deviations*, called by the sta- 
tistician standard deviations, of A, B, C, respectively, due to errors of 
sampling. They may be found approximately by using the values of B and C 
obtained by the approximate method. Thus, by the ordinal^ theory of en-ors, 



4 



7*2^2 



[{sl^i^) iZjyf+islZf) CZiyf] 



hn^si 



'-[(Zjyr+iZiyr]Ii' 



where Sr = \ — Z'''^> ^^^ t^iis also equals the mean square deviation of 

the z's, about the mean plane. Solving this equation for i* and substituting 
in (8), one gets 

* One might as well use "probable" deviations, or probable errors, by multiplying: the 
standard deviations by the appropriate constants. 



102 



B. H. CAMP. 



(9) 



dB = 



2jer 
Zdr 






hksrYiZjyy + iZiyf ]/Zi' 



Now e and r are uncon-elated, and so the most pro bable va lue of their coef- 
ficient of correlation, ^ er/seSrn, where Se = l/^e*/n, is zero, and it is 

almost certain* that this quantity does not depart fi-om zero by more than 
three times its own standard deviation; it is probable that this departui-e is, 
in fact, less than the standard deviation. Thus, almost certainly,* 



,er 



nSe Sr 



VW' 



and therefore | ^ er | -c 3 Sr Se Vw"; and similarly | ^ dr | < 3 Sr Sd V~n. 
On substitution in (9), it is found that, almost certainly ,t 



dB\<.Zt 



'K Si*" 



Se I Zjy \'k-\-Sd\Ziy\h 

hh]/{Zjyf+{Ziyf ' 



An approximation to this expression may be made by putting y = F — To, 
since Y — To is small. The result may then be simplified by using the fact 
= Zj = 0, and inserting the values of h and h from (4). Finally, then, 



that Z -i 



\dB\<Zss\nZi* 






Z^Tl 



ZiY 



Sd 



(10)t 



Sb \ »* 



dCl-^Zsu V'n'Zi'' 



IZiXl ^ , \ZjX 



Sd 



yiZixf+iZjxf 

(lA = — — (dBZX+dCZY). 



n 



* On the assumption of a Gaussian distribution, the probability that the coefficient of 
correlation will exceed its standard deviation is 0.S2, and the probability that it will exceed 
three times its standard deviation is only O.0027. 

t See preceding foot-note. In the examples following the theory, limits obtained from (10) 
are described as "nearly certain". Limits obtained from (10) without the factor 3 are described 
as "probable". 
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These formulae hold good only if the i and j sequences are chosen as in (2) 
and (3). All the sums involved occui* in the normal equations as determined 
by the short method. After these sums have been found, a slide rule com- 
putation suffices for dB/s^, and dCISf.. It is these ratios, and not the absolute 
values of the errors which are important in determining the question whether 
the short method is justified. For example, if A = dBIs^ is as small as O.oi, 
then there is a probability of O.ese that dB, the error introduced by the short 
method, is numerically less than the necessary error in B, due to the accidental 
enors of observation. The corresponding probabilities for other A's are given 
by a Gaussian table, as follows: 



; 


prob. 


,! 


prob. 


X prob. 


.008 


.996 


.10 


.92 


.4 .69 


.01 


.992 


.20 


.84 


.5 .62 


.05 


.957 


.30 


.76 


1.0 .32 



Similar remarks apply to C. As will become obvious from the examples to 
follow, it is necessary to have only an approximate estimate of the sizes of Se 
and Sd- These are sometimes obvious from the nature of the problem (e. g., 
Ex. 2 below). In other cases they must be computed. The labor of doing this 
is slight. Consider, for example, Sg. The values of ^ iX and ^ «" from which 
h is found have already been obtained in the short solution. Then ei = Xi 
— Zo — hi, si ^"^ 6*171. It follows from the last equation of (10) that dA 
may be made zero, whether the X's and Y's are nearly eqiii-s][>aced or not, 
provided they are so chosen that ^ X = ^ F = 0. This is a consideration 
worth keeping in mind if A is the quantity most desired from the observations. 
It follows also that, if ^ XI n and ^ Yin are small, the approximate solution 
for A may be used, instead of the long solution, with small en-or. It seems to 
the author that this method could be used to advantage in getting the time 
from transit observations. Tables already in use in connection with other 
methods make it easy to select the stars properly (Cf. Ex. 3 below). 

The expressions just found are available after the approximate noi-mal 
equations have been found. It is now desirable to shorten them by further 
approximations so that they may be made to yield good estimates of the errors, 
in advance of any computation. It will be necessary to assume now that the 
order of the Z's is uncorrelated with the order of the T's. This is frequently 
the case, but there are important instances in which it is not, and in such 
instances the subsequent approximations must not be used. Approximately, 
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(11) Z i (Y- Yo) = Zj iX- Xo) = 0, Xi -Xo = ill, Yj - To =jk. 

Omitting the coefficients 3 in (10), and substituting (11), one obtains, as 
a provisional value for dB, 



(.2) « = ^|/^. 

Ifmisodd,^i* = w(«— l)(n + l)/12. If wiseven,^?* = n(«— l)(n + l)/6 
but in the latter case h and Ic are only half what they are in the former case, 
to that, if H and K be approximately the average intervals between the Z's, 
and between the Fs, respectively, then in all cases: 

(13)*li^— «« 2K3 \dC\ __ sa 2K3 



, H y(n+l)(« — 1)' Sc K l/(n + l)(w — D* 

The quantities SelH, sal K, measure the divergences from equi-spacing, and it 
is natural than they should appear as factors in the expressions for the errors. 
In most problems where any real attempt is made at equi-spacing it is possible 
to make !>elH and sal K as small as 1/3. Indeed, ten times this smallness is 
to be expected frequently. In very bad cases, as in Example 3, where the 
conditions of the problem are such as to interfere with equi-spacing, these 
quantities may be as large as 1.5. It follows from (13) that dBls^ and dCISf, 
approach zero with 11 n. One must compute dA from the formula of (10). 

3. Summary. Let dA,dB,dCh& the errors made in ^, jB, C if the short 
method of § 1 is used in connection with (1) instead of the exact least square 
solution. To find dB/s^ and dC/Sf,, provisionally, substitute in (13), s^ and s^ 
being the mean square deviations! of B and C due to the fluctuations of 
the observations themselves. The values of Se/S andsd/Z" may be taken 
as stated under (13). To find dAlsj^, then use the last equation of (10). 
Formulae (13) are independent of the particular type of ascending equi-spaced 
sequence of integers used as multipliers, but formulae (10) are not. 

To find closer approximations to these relative errors, after the short com- 
putation has been made, use equations (10), noting what is said in the re- 
mainder of the paragraph containing (10). ^ i* is best found from a short 
table which the computer may as well make for himself, although several such 
tables are published for large values of n|. 



* Probable values. Insert the factors 3 to obtain more certain limits. See preceding 
foot-note. 

tAlso called "standard deviations". 

% E. g., "Tables for statisticians and biometricians", by Karl Pearson, Cambridge University 
Press, 1914. 
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For the special case Z ^ BX-\- CY, the same formulae may be used if, 
and only if, Xo and Fo are numerically small, relative to those values of X and Y 
which are distant from Xo and To. Otherwise, it is not recommended that 
the short method be used. 

For the special case Z = A-\- BX, formulae (10) and (13) take on the 
simple forms: 



(10a)* 



(13a)* 



\dB\ 



= K»2'--^ 



dA = —dB 



Zx 



n 



\dB\ 



St 



2Vd 



H l/(n + l)(» — 1)' 



and the various remarks made in coimection with (10) and (13) apply of course 
to these special forms. 

4. Examples. Example 1. For the first illustration a rather bad case Avill 
be chosen, a group of five stars composing a time set, with only the crudest 
attempt at equi- spacing. They are chosen fi'om the unweighted set in 
Campbell's "Elements of Practical Astronomy", page 150, and in the following 
observational equations, the letters A, B, and C, correspond with Campbell's 
X, c, and a, respectively. The equations are 

— 1.6« = A — 3.8li J5+3.S96 C 

— O.IS = A-\- 1.007 B + 0.S87 C 
+ 0.98 = ^+2.915 5—1.888 C 

— Ooil = A—1.1U JB— 0.887 C 
+ 0.47 = ii — 4.898 jB— 2.875 C 



The results, as obtained by the exact method of least squares and also by the 
approximate method are: 





True 
Values 


Approximate 
Values 


Differences, 

called "errors" 

above 


Standard 
Deviatious 


A 
B 

C 


.0488 

.1266 

— .8668 


.0612 

.1289 

— .3689 


.0024 
.0023 
.0030 


.024 
.008 
.011 



Probable values. See foot-notes to (10) and (13). 
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The next table gives the provisional and final estimates of the maximum values 
of the errors, as derived from the formulae of this paper, also their true values. 
In using equation (13) it was assumed that sJH = salK = 1/3. It turned out 
that these numbers were O.27 and Cse, respectively. From the table in § 2, 
it follows that, assuming a normal distribution of error, the fluctuation of A 
due to the observations themselves, would, in 92 cases out of 100, exceed the 
difference (.oos*) introduced by the short method of computation, and the 
corresponding numbers for B and C are 77 and 78 cases. It would seem to 
the author that, in examples like this, the short method is usually justifiable, 
but that it might not be sometimes. It is, however, not the object to present 
an argument for the short method, but to present a criterion by which the 
error to be committed by it may be estimated. 





Limits* by (13) 


Limits by (10) 


True 
Values 


Probable 


Kearly 
Certain 


Probable 


Nearly 
Certain 


1 dAjs^ 1 
1 dBlB„ 1 
1 dCIsc 1 


.24 
.24 


.72 
.72 


.20 

.26 


.60 
.75 


.104 
.287 
.282 



Example 2. A very good case will now be considered, a set of observations 
used by Roesert with the equi-spacing disturbed by the arbitrary addition of 
small quantities. These small quantities were chosen at random from a table 
of numbers subject to the limitation that they should lie between .05 and 
— .05. The equations and tables following have interpretations similar to 
those of Example 1. It is obvious from a mere glance at the equations that 
Se/ifis not greater than about .004, and this was the value used in the pro- 
visional estimate. The true value is . 0028. If this had been used instead of 
0.004, the provisional estimate for dBls^ would have differed from the true 
value by only about 4 per cent., thus illustrating the closeness of the inequality. 
It is not desirable, however, in practice to compute sjH in advance, and 
accordingly the rough estimate (O.004) is used. With this rough approximation, 
formula (13) shows that the value of dB/s^ will be so small as to make the 
difference between the two methods entirely negligible. This is exhibited in 
the table below. 



• The first column is obtained from (13). The second column is three times the first. 
See also foot-note to (10). 

t Loc. cit., p. 371. See also Weld, Theory of errors and least squares, 1916, p. 76, 
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0.ii« = .l+10.c5 B, 

0.20S = . 1+19. 95 B, 
0.295 =.1 + 29.99 5, 
0.388 = A + 40.01 B, 
0.503 =^ + 49.97 B, 



0.595 =.4.+ 60.01 B, 
0.675 = A+ 69.98 B, 
0.760 =i4+ 79.98 .B, 
0.850 = .4+ 90.</4 jB, 
0.926 = 4+100.01 .B. 





True 


Approximate 


Difference 


Standard Deviation 


B 


.028478 .028425 
.0091417 .0091418 


.000053 
.0000001 


.00315 
.00011 





Limits by (13 a) 


Limits by (10 a) 


True 


Probable 


Nearly 
Certain 


Probable 


Nearly 
Certain 


dAls^ 

dBiB, 


.00139 


.00417 


.00099 


.00297 


.00168 
.00096 



A is not so well determined as B. This is because the origin is far from the 
middle X. 

Example 3. Finally, a very bad case will be considered, viz., Campbell's 
eleven weighted equations* out of which the data of Example 1 were selected. 
Here the method of choice of stars was such as deliberately to prevent equi- 
spacing, as a rough plotting of U, X, and Fwould show. The typical equation is 

Z = AU-\-BX+CY, 



a type not considered in the theory, but formulae (13) are valid. lu such bad 
cases as this, the divergence from equi-spacing is so great that the ratios like 
St/H, which measure it, cannot be assumed small, and so it is advisable to 
estimate them by a graph. If, for example, Xi be plotted as a function of /, 
it will be seen that s« is about unity and H about 2/3, making Se/H about 1 . s. 
The corresponding fractions for U and T may be found in a similar manner 
to be about 1 . t and 1 . o, respectively. 



* Loc. cit, p. 155. 
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True 


Approxi- 
mate 


Difference 


Standard 
DeTiation 




Limits by (13) 


True 


Probable 


Nearly 
Certain 


A 
B 
C 


.058 

.115 

— .883 


.058 

.122 

— .886 


.000 
.007 
.008 


.017 
.012 
.020 


dA/s^ 
dB/sg 
dC/sc 


.39 

.47 
.82 


1.2 
1.8 

1.0 


.01 
.68 
.15 



Thus, even in this case, the provisional formula (13) is adequate to show that 
the errors are probably as small as half the standard deviations of the 
quantities measured. The very small actual error in the time correction. A, 
is due in part to the fact that the means of the X's and Y's are small. This 
example, taken together with Example 1, also serves to illustrate how the 
disturbance due to bad spacing can be masked by an increase in the number of 
equations of condition. This result is to be expected from the form of (13), as 
was noted in the sentence preceding the last of § 2. The value of the short 
method in saving labor increases greatly with the number of equations of con- 
dition, and it is therefore particularly comforting to know that at the same 
time the error in using it diminishes. 

5. Conclusion, The short method works sufficiently well to justify its use, 
certainly in cases as good as Example 2, probably in cases like Example 1, 
and perhaps (depending on the accuracy desired) in cases as bad as Example 3. 
In any case the goodness of the method may, with sufficient accuracy, be 
judged in advance by the criteria (13) or (13a). When the short method is 
used, the gain in time is very great, especially if one employs a computing 
machine. The author used an electrically driven listing machine, but an 
ordinary adding machine would have done almost as well. All the twelve 
coefficients of the normal equations in Example 3 for the eleven equations of 
condition were found in this way in less than twenty minutes. In the short 
method, all the multiplications necessary for the coefficients like ^ iX are 
by small integers. Thus, with an adding machine, the products, iX, may be 
found and added at the same time. 

Wesleyan Univeespty, 
May, 1922. 



